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Risk  maturity  model 

The  new  approach  is  based  on  the  concept  of  five  levels  of  Risk 
Management  maturity.  These  depict  the  evolution  of  risk  management 
capability  resulting  from  the  actions  of  management  and  the  investment 
in  enterprise  risk  management  frameworks,  systems,  people  and 
processes 


Example  -  high  level  characteristics  for  each  maturity  level  for  ‘Policy  and  Objectives’  element 
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letrics  to  demonstr 
'eturn  on  investmei 


ale 


classe: 


CANADIAN  HUMAN  RIGHTS  COMMISSION 


HUMAN  RIGHTS  MATURITY 
MODEL  CONTINUUM 


MANAGED  AND 
ROUTINE 

Proactive  approach  to  address 
human  rights  matters. 


PREDICTABLE  AND 
SUSTAINABLE 

Day-to-day  operations  and 
practices  integrate  human  rights 
principles. 


LEVEL  4 


CONTINUOUSLY 

OPTIMIZING 

A  culture  of  human  rights  is 
incorporated  in  day-to-day 
operations  and  is  continuously 
improving. 


LEVEL  5 


DEFINED 

Implementation  of  a  structured 
approach. 


LEVEL  3 


INITIATED 

Initial  steps  taken  to  create  a 
human  rights  culture, 


LEVEL  2 


LEVEL  1 


*  Leadership  commitment  to 
culture  change, 

*  Consultation  and 
communication  begins. 

*  Basic  legislated 
requirements  are  met. 

*  Adequate  capacity  and 
resources. 

*  Basic  quantitative  data 


-  Management  engaged  in 
culture  change. 

•Structure  in  [place  and 
communicated  to  staff. 

■  Policies  are  implemented 
and  discrimination 
complaints  process 
established. 

•  Resources  committed  to 
implement  HRMM, 

•  Qualitative  data  collection. 


-  Management  acts  in 
accordance  with  their  roles 
and  responsibilities 
regarding  human  rights, 

•  Proactive  communication 
and  consultation  involving 
the  whole  organization. 

•  Multi-disciplinary  approach 
involving  all  sectors  of 
business, 

•  Proactive  systems  in  place 
to  manage  human  rights 
issues. 

•  Development  of  a  human 
rights  performance 
measurement  framework. 


•  Human  rights  roles  and 
responsibilities  are  acted 
upon  throughout  the 
organization. 

•  Organization  has  built 
relations  with  external 
partners  with  respect  to 
human  rights. 

•  Ongoing  consultation 
with  external  partners,  key 
stake  holders  to  promote 
human  rights  principles. 
•Internal and  external 
policies  and  practices 
reflect  human  rights  and 
are  shared. 

•  Enhancement  and  sharing 
of  performance 
measurement  framework. 


•  Leadership  demonstrates  its 
broad  commitment  to  human 
rights. 

■  All  levels  of  the  organization 
share  responsibility  for 
human  rights. 

•  Broad  promotion  of  human 
rights  principles. 

■  Policies  and  processes  foster 
human  rights  in  areas  beyond 
economic  interests. 

-  Performance  measurement 
framework  incorporates 
parameters  related  to  the 
promotion  of  human  rights. 
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Health  Care  Network  Maturity  Model: 

Posted  on  July  1 5,  2013 

by  Paul  D.  Taylor,  M.D.,  CM  to,  Wellcentive,  Inc. 

Time  and  Tide  Wait  For  No  Man 
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MANCEAMD  IMPROVEMENT 


Affiliated 

Documentation 

■  Implement  EMR 
-  Collect  data  at 

Point  of  Care 

■  Focus  on  episodic  care 


Coordinated_ High-Performing 

Collaboration  &  Improvement 


Engaged 

Organization  &  Measurement 

■  Aggregate  and 
Normalize  data 

■  Engage  Providers 
*  Measure  against 

Payer-driven  programs 

■  View  Community  Info 


Target  high-value 
|  opportunities 

■  Prioritize  high-risk  patients 

*  Initiate  care  management 

*  Identify  gaps  in  care 
-  Patient  outreach 

*  Closed  loop  analysis 


Optimize  Clinical  and 
Financial  Outcomes 

Utilize  predictive  modeling 
■  Assess  organizational  risk 

*  Manage  cost  &  utilization 

-  Enhance  contract  positioning 

*  Improve  the  patient  experience 


|  Fee  for  Service 

Pay  for  Performance 

Shared  Savings  & 
Bundled  Payments 

Shared  Risk 
&  Capitation 

Risk 


Customer  Engagement 


STAGES 


CXMM: 


IGNORED 


Bu  s  i  ne$$  i  S  i  award- 1  poking. 
Has  only  a  basic  understand¬ 
ing  of  (and  interest  in)  who 
customers  are  or  what  they 
want.  Customers  often 
believe  the  business  doesn't 
understand  or  care  about 
them.  Customer  experience 
is  inconsistent  and  often 
unpleasant. 


THE 
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STAGE 


UNDERSTOOD 


STAGE 


HEARD 

Business  has  a  good 
understanding  of  who 
customers  are  and  how  they 
fed.  and  uses  this  insight  to 
make  adjustments  to  the 
customer  experience. 
Customers  may  believe  the 
business  i$  interested  in 
learning  from  them,  but  they 
don't  have  much  attachment 
to  the  brand. 


Business  has  programs  that 
drive  deep  insight,  track 
customer  preferences,  and 
ensure  a  consistent 
experience.  Customers 
believe  their  needs  are 
mostly  addressed  by  the 
products  and  services 
offered.  There  is  a  clear 
linkage  between  customer 
insight  and  products. 


MATURITY 


OF 
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STAGE 


I 


E>  STAGE  i 

LB 

PASSIONATE 

Business  has  such  strong 
relationships  with  custom- 


ENGAGED 

Business  has  a  comprehen¬ 
sive.  act ionabie  picture  of 
customers,  and  a  culture  of 
accountability.  This  gives  it 
differentiation  in  the  market 
and  generates  loyalty. 
Customers  believe  the 
business  cares  about  them, 
and  they  trust  the  company. 
Customers  demand 
increased  value,  and  they  are 
rewarded  for  their  loyalty. 
They  are  willing  to  spend 
more  for  the  assurance  of  a 
consistently  positive 
experience. 


ers,  it  has  become  the 
undisputed  industry  leader 
in  Net  Promoter  Score  and 
customer  retention. 
Customers  are  passionate 
evangelists.  They  feel 
privileged  to  associate  with 
the  company  and  share 
stories  of  their  positive 
experiences  with  others. 


Business 
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software 


Governance 
and  recovery 
activities  are  ad 
hoc,  improvised 
and  reactive 


Knowledge, 
responsibilities 
and  skills  are 
lacking 


program 
ementor 
i  cwVery  plan 
automation 


Management 
processes 
support  event 
response  only 


ITDRM 

responsibility 
assigned  ” 


Level  3 
Defined 


No  program 
management  or 

recovery  plan 
automation 

Vision  and 

program  strategy 
definition  in 

progress 

BCM  roles, 
responsibilities 
and  steering 
committee  in 

place 


Activities  are 
IT-centric;  no 
recovery 
classes 


Awareness 
triggered  by  a 

disaster  event 


Activities  are 
IT-centric; 

basic  recovery 
classes  and 
plans 

Limited  business 
involvement  and 
commitment 


ITDRM 
classes  and 

plans  for  all 
mission- 
critical 
applications 

Recovery 
expectations  and 
delivery  are  better 
aligned 


Level  4 
Managed 


Level  5 
Optimizing 


Program 
management  and 
recovery  plan 
automation  in 
place 

BCM  processes 
standardized  and 

exercised  across 
enterprise 


governance 

is  formalized 

IT  DRM  classes 
and  plans  cover 
more  than 
mission-critical 
applications; 
business 
recovery  plans 
in  place 

Recovery 

expectations  and 
delivery  are 
aligned 


Program 
man  a  gem  ent 
autom  ati  on  en  ab  les 

continuous 

improvement 


t 

KRIsand 
KPIs  linked 
and 
reported 

BCM  program 
responsibility 
aligned  with 
strategic 
business 
management 

Comprehensive 
BCM  plans  are 
in  place  and 
regularly 

exercised 

Recovery 

expectations  and 
delivery  are 
aligned 


DevOps 


Collaboration 

Automation 

Process 


Poor,  ad-hoc 
communication  and 
coordination 

No  automation 


jed 

communication, 
some  shared 
decision  making 

Siloed  automation, 
no  central 
infrastructure 


Unpredictable, 
uncontrolled 
reactive  processes 


Processes  are 
managed  but  not 
standardized 


f 

Collaboration, 
shared  decision 
making  and 
accountability 

Central  automated 
processes  across 
the  application 
lifecycle 

Processes  are 
standardized 
across  the 
organization 


Collaboration- 
based  processes 
are  measured  to 
identify 

inefficiencies  and 
bottlenecks 
Collect  and 
analyze  metrics  of 
the  automated 
processes  and 
measure  against 
the  business  goals 


Visibility  and 
predictability  of 
entire  process 
quality  and 
performance 


Effective 
knowledge 
sharing  and 
individual 
empowerment 


Self-service 
automation,  self¬ 
learning  using 
analytics  and 
self-remediation 


Process  risk  and 
cost  optimization 
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Social  Media 


listening 


*  REACTIVE  AND  TAKEN  UNAWARE  BY  SOCIAL  MEDIA 


broadcasting 


*  FACEBOOK  AND  TWITTER  PRESENCE 

*  BROADCAST  STANDARD  MARKETING  VIA  SOCIAL  MEDIA 

*  TARGETED  TO  SPECIFIC  INDIVIDUALS 

*  OBJECTIVE  ISSUES  AT  POINT  OF  NEED 


marketing 


*  SOCIAL  MEDIA  STRATEGY 
-  BRAND  DASH  BOARDING 

*  ENGAGEMENT  MARKETING 

*  MINIMAL  CUSTOMER  CARE  INVOLVEMENT 


customer  care 


*  SCALABLE  ENGAGEMENT  PROCESS 

*  SHARE  BRAND  +  PERSONALITY 

*  MANAGED  PROCESS 

*  TEAMS  WORK  QUEUES  +  GENERATE  REPORTS 


proactive  engagement 


*  PROACTIVE  CUSTOMER  CARE 

*  CREATE  CONTENT  TO  HELP 
CUSTOMERS  ACHIEVE  THEIR 
GOALS 

*  SOCIAL  MEDIA  BUSINESS 

*  PROACTIVE  SALES 
-  INTELLIGENCE 


total  immersion 


*  ENTIRE  COMPANY  PARTICIPATES  IN  SOCIAL  MEDIA 
CUSTOMER  CARE 


Objectives  of  This  Session 

Maturity  models  are  effective  tools  for  improving  an 
organization’s  security  capabilities  and  outcomes. 

But  knowing  which  model  to  use  and  how  to  use  it  is 
paramount  to  success. 

•  Improve  your  understanding  of  maturity  model  concepts 

•  Learn  about  the  use  of  maturity  models  by  examining  recent 
examples  in  the  cybersecurity  and  resilience  domains 

•  Be  aware  of  caution  flags  when  dealing  with  maturity  models 

•  Determine  how  to  choose  the  right  model  for  your  specific 
needs  (improvement  vs.  assessment,  etc.) 


Software  Engineering  Institute 


Carnegie  Mellon  University 
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Overall  Outline  of  This  Session 

Setting  the  Stage 
Background  and  History 
ABCs  of  Maturity  Models 
Panel  Discussion 
Closing  Thoughts 

CERT  ^ — -  Software  Engineering  Institute  Carnegie  Mellon  University 
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Maturity  Models  Member  Query 


i 


Have  you  or  your  organization  ever 
used  any  type  of  maturity  model? 


If  yes: 


1 

■  ■  |A  _  CC 


‘mm. 


w  K"J- 


’  "  C-  1  £Ji  ■ 


2 

In  what  areas? 

3 

For  what  purposes? 

4 

What  were  the  reasons? 

5 

Which  maturity  models? 

If  no: 

6 

How  do  you  assess  the  maturity  of  your 
cybersecurity  program? 

'CERT  * 
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Maturity  Models  Member  Query  -  Q1 


Have  you  or  your  organization  ever  used  any  type  of 
maturity  model? 


0  5  10  15  20  25  30  35 


CERT  Software  Eng  ineer  ing  Institute  Carnegie  Mel  Ion  University 


Maturity  Models  Member  Query  -  Q2 


In  what  areas? 
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Cybersecurity  /  Information  Security 

Risk  Management 
IT  Operations 
IT  Management 
Software  Engineering 
Disaster  Recovery  or  Business- 
Process  Management/Improvement 
Other  (Please  Specify): 
Systems  Engineering 
Resilience  Management 
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20 
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Maturity  Models  Member  Query  -  Q2 

In  what  areas? 

J  OTHER: 

•  Client  specific  projects 

•  IT  architecture 

•  Incidence  response 

•  Identify  and  access  management 

•  Product  development 

•  Roadmap  activities 

•  Access  one’s  ability  to  deal  with  risk 

•  Build  best  practices 

•  As  a  very  large  company,  the  use  of  maturity  models  varies  greatly  not 
only  from  area  to  area  but  also  from  group  to  group  even  within  the 
same  area. 

CERT  ^ — -  Software  Engineering  Institute  Carnegie  Mellon  University 
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Setting  the  Stage 

•  The  need  for  “measuring”  operational  activities  &  their  effectiveness 

•  Are  we  doing  the  right  things? 

•  Are  we  using  the  right  tools  to  measure? 

•  Are  we  measuring  the  right  things? 


Today’s  Operating  Environment 


Rapid  changes  in  technology 
and  its  application  in  a  wide 
range  of  industries. 


Introduction  of  many  new 

systems,  business  processes, 
markets,  risks,  and  enterprise 
approaches. 


Many  immature  products 
and  services  being 
consumed  by  enterprises  that 
themselves  are  in  a  state  of 
change. 


CERT 
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Challenges  at  Hand 


How  can  you  tell  if  you  are  doing  a  good  job  of  managing  these  changes? 


What  are  effective  ways  to  monitor  your  progress? 


How  do  you  manage  the  interactions  of  systems 
and  processes  that  are  continually  changing? 


How  do  poor  processes  impact 
interoperability,  safety,  reliability, 
efficiency,  and  effectiveness? 
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Which  Tool  Should  I  Use? 


Your  organization  wants  to  know  SOMETHING  about 
your  mission  operation: 

•  How  EFFECTIVE  are  we? 

•  Do  we  have  the  right  SKILLS  and  CAPABILITIES? 

•  Do  we  have  the  right  TECHNOLOGIES? 
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Observation 


The  development  and  use 
of  maturity  models  in  security, 
continuity,  IT  operations,  & 
resilience  space  is  increasing 
dramatically. 


♦>  ♦>  ♦> 


Do  Maturity  Models  Measure  the  Right  Thing? 


May  not  measure  what  you  think  it  measures 

>  Practice  maturity  vs.  organizational  maturity? 

May  give  you  inaccurate  data  on  which  to  base  decisions 

>  Process  performance  vs.  product  performance? 

Can  increase  cost  without  increasing  benefit 

>  An  improved  process  may  not  result  in  compliance 

May  provide  a  false  sense  of  confidence 

>  A  robust  process  may  not  improve  malware  management 
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CMU  -  SEI  -  CERT® 


Carnegie 

Mellon 

University 


Software  Engineering  Institute  (SEI) 

•  Federally  funded  research  and  development  center 
based  at  Carnegie  Mellon  University 

•  Basic  and  applied  research  in  partnership  with 
government  and  private  organizations 

•  Helps  organizations  improve  development, 
operation,  and  management  of  software-intensive 
and  networked  systems 

CERT®  -  Anticipating  and  solving  our 
nation’s  cybersecurity  challenges 

•  Largest  technical  program  at  SEI 

•  Focused  on  internet  security,  digital  investigation, 
secure  systems,  insider  threat,  operational 
resilience,  vulnerability  analysis,  network  situational 
awareness,  and  coordinated  response 
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Cyber  Risk  and  Resilience  Management  Team 


Engaged  in 

•  Applied  research 

•  Education  &  training 

•  Putting  into  practice 

•  Enabling  our  federal,  state,  and  commercial  partners 

In  areas  dealing  with 

•  Maturity  models 

•  Operational  resilience 

•  Resilience  management 

•  Operation  risk  management 

•  Cybersecurity  maturity  models 

•  Integration  of  cybersecurity,  business  continuity,  &  disaster 
recovery 
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Background  and  History 

•  Where  do  maturity  models  come  from? 

•  Early  development  and  instantiation 


In  the  Beginning  There  Was  “Quality  is  Free” 


QUALITY 
IS  FREE 

The  Art  of  Making  Quality  Certain 


Howto 

manage 

quality— 

so  that  it 
becomes 
a  source  of 
profit  for 
your  business 


PWTT.TP  R  frcnsRY 

author  of; 

"Tlh©  Art  -ol  Getting  Your  Own  Way" 


Viewed  “quality”  as  a 
characteristic  owned  by 
everyone  in  the  organization 

Created  the  Quality 
Management  Maturity  Grid  to 
express  organizational  maturity 
across  a  range  of  quality 
attributes  or  categories 

Defined  observable  outcomes 
as  benchmarks 
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The  Quality  Management  Maturity  Grid 


Quality  Management  Maturity  Grid  (Crosby)  Assessor:  Department: 

Measurement  Categories 

Stage  1 :  Uncertainty 

Stage  2:  Awakening 

Stage  3:  Enlightenment 

Stage  4:  Wisdom 

Stage  5:  Certainty 

Management 
understanding  and 
attitude 

No  comprehension  of 
quality  as  a  manageme 
tool  Tend  to  blame  qua 
department  for  ''quality 
problems'1. 

nt 

ity 

Recognising  that  qualify 
management  may  be  ol 
value  but  not  willing  to 
provide  money  or  time  1 
make  it  all  happen. 

r 

to 

While  going  through 
quality  improvement 
programme  learn  more 
about  quality  manager™ 
becoming  supportive  ar 
helpful 

ent; 

id 

Participating  Understand 

absolutes  of  quality 
management  Recognise 
their  personal  role  in 

continuing  emphasis 

r~ 

Consider  quality 
management  as  an 
essential  part  of  company 
system. 

Quality  organisation 
status 

Quality  is  hidden  in  • 

manufacturing  or  l 

engineering  departments  • 
Inspection  probably  not  • 

A  stronger  quality  leader  is 
appointed  but  main 
emphasis  is  still  on 
appraisal  and  moving  the 

appraisal  is  incorporated  1 
^nd  manager  has  role  in  1 

Observable  a 
characteristic 

ttributes  or 

:s 

part  of  organisation. 
Emphasis  on  appraisal 
sorting 

• 

an<3 

• 

• 

product.  Still  part  of 
manufacturing  or  other 

Jnanagement  of  compar 

• 

• 

• 

• 

• 

iy- 

Involved  with  customer 

affairs  and  special 
assignments 

Problem  handling 

Problems  are  fought  as 
they  occur;  no  resolutic 
inadequate  definition;  ic 
of  yelling  and  accusatic 

m 

3tS 

ms 

Teams  are  set  up  to  att? 
major  problems.  Long- 
range  solutions  are  not 
solicited 

ack 

Corrective  action 

communication 

established.  Problems  £ 
faced  openly  and  resoh 
in  an  orderly  way 

ire 

/ed 

Problems  are  identified 
early  in  their  developme 
All  functions  are  open  ti 
suggestion  and 
improvement. 

M 

0 

Except  in  the  most  usue 
cases,  problems  are 
prevented 

il 

Cost  of  quality  as  %  of 
sales 

Reported:  Unknown 
Actual:  20% 

Reported:  3% 

Actual:  13% 

Reported:  3% 

Actual:  12% 

Reported:  6.5% 

Actual:  3% 

Reported:  2.5% 

Actual:  2  5% 

Quality  improvement 
actions 

No  organised  activities 
understanding  of  such 
activities 

No 

Trying  obvious 
'motivational'1  short-rani 

efforts. 

ge 

Implementation  of  a  mu 
step  programme  (e  g 
Crosby's  14-step)  with 
thorough  understanding 
and  establishment  of  es 
step. 

Iti- 

1 

ich 

Continuing  the  multi-ste 
programme  and  starting 
other  pro-active  / 
preventive  product  qual 
initiatives 

P 

J 

ity 

Quality  improvement  is 
normal  and  continued 
activity 

a 

Summary  of  company 
quality  posture 

MWe  don't  know  why  we 
have  problems  with 
quality". 

"Is  it  absolutely  necessi 
to  always  have  problem 
with  quality?'1 

ary 

s 

'Through  management 
commitment  and  quality 
improvement  we  are 
identifying  and  resotvini 
our  problems.'1. 

3 

'  Defect  prevention  is  a 
routine  part  of  our 
operation." 

NWe  know  why  we  do  n 
have  problems  with 
quality.'1 

ot 
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Evolution  of  the  QMMG 


1986  -  Watts  Humphrey  formalizes  the  Process  Maturity 
Framework  into  the  Capability  Maturity  Model  for 
Software  (SW-CMM)  at  Carnegie  Mellon’s  Software 
Engineering  Institute 

Driven  by  USAF  need  to  measure  capabilities  of  software 
contractors 


Architecturally  based  on  the  QMMG 
but  reflective  of  observed  best  practices 
for  software  development 

2000  -  CMM  Integration  (CMMI) 
created  to  combine  software, 
systems  engineering  and  integrated 
product  processes;  now  at  vl  .3 
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CMMI 


for  Development 


for  Pfottis 
Integration 

Jild  Pc'Cidicel 
I  mpruvHiKn? 

Tncnin  Eimtiein 


Mary  ClixiEMj 
ikv  Kmir.Kl 


ABCs  of  Maturity  Models 

•  What  are  maturity  models? 

•  Types  of  maturity  models 

•  Examples  of  maturity  models 


Maturity  Model  Defined 


An  organized  way  to  convey  a  path  of 
experience,  wisdom,  perfection,  or  acculturation 

Depicts  an  evolutionary  progression  of  an 
attribute,  characteristic,  pattern,  or  practice. 

The  subject  of  a  maturity  model  can  be 

objects  or  things,  ways  of  doing 
something,  characteristics  of 
something,  practices, 
controls,  or  processes. 


r*  - 
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Maturity  Models  Provide... 

Means  for  assessing  and  benchmarking  performance 
Ability  to  assess  how  a  set  of  characteristics  have  evolved 
Expression  of  a  body  of  knowledge  of  best  practices 
Means  to  identify  gaps  and  develop  improvement  plans 
Roadmap  for  model-based  improvement 
Demonstrated  results  of  improvement  efforts 
Common  language  or  taxonomy 
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Maturity  Models  Member  Query  -  Q3 


&■ 


a 


y>- 


■rti  ■ 


For  what  purpose? 

'  »  p  -  -  ifr-'  '  m  - .■  •  ■  ' m  - p  -  -  :-i 

■ 1  '&>-  ■  e*  ^  ‘  ‘  "  >. 1  (fr  '  ^ 

l  ■  •  i"hk  'r  .,■  _ ,  ■  -fW  ^  .■  ■_  -__  ,-h. 

To  identify  gaps  and  shortcomings  in  certain  areas 

To  establish  (improvement)  goals  to  achieve 

To  assess  or  measure  current  state  of  certain 
characteristics  or  capabilities 

To  develop  new  and/or  improved  capabilities 

As  a  mean  to  introduce  a  common  vocabulary  and 

nomenclature 

Other  (Please  Specify): 


0  5  10  15  20  25  30 


r\ 
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Maturity  Models  Member  Query  -  Q3 

For  what  purpose? 

I  OTHER: 

•  Governance 

•  To  compare  to  other  organizations 

•  Yes  to  all  with  emphasis  on  common  vocabulary  and  driving 
i  to  goals. 

•  Define  strategic  IA  maturity  objectives  and  develop  an 
action  plan  for  improvement 

•  Yes  to  all  but  the  approaches  vary  considerably  across  the 
company 

&  fa  ^  ■{£0k , .•* fa  ®  ^  fa  &  &  ^  fa  ^  ^ ®  ^ 
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Maturity  Models  Member  Query  -  Q4 


For  what  reason? 

mm 

It  was  determined  to  be  the  best 
approach;  It  was  the  right  thing  to  do 

For  competitive  advantage 
Other  (Please  Specify): 

To  test/evaluate  the  approach 

To  comply  with  some  national  or 
international  standard 

Required  by  some  local  or  federal 
policy  or  legislation 


0 
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Maturity  Models  Member  Query  -  Q4 

For  what  reason? 

I  OTHER: 

•  To  help  create  strategy 

•  To  develop  capability 

•  To  test  and  evaluate  approach 

•  To  communicate  upwards 

•  To  set  expectations 

•  To  communicate  opportunity  for  improvement 

•  Mandated  across  UK  Government  Departments 

•  All;  depending  upon  area  of  the  company  and  various  contract  drivers. 

•  A  combination  of  drivers  towards  pragmatic  centralized  management 
and  scoring. 

•  Trying  to  establish  a  common  method  to  develop  roadmaps 
understandable  by  executive  committee  and  board  of  directors 


Key  Components  of  a  Maturity  Model 


g? 


Levels 

•  The  measurement  scale 

•  The  transitional  states 

Domains 

•  Logical  groupings  of  like  attributes  into  areas  of  importance 
to  the  subject  matter  and  intent  of  the  model 

•  Logical  groupings  of  like  practices,  processes,  or  good 
things  to  do 

Attributes 

•  Core  content  of  the  model  arranged  by  domains  and  levels 

•  Typically  based  on  observed  practices,  standards,  or  expert 
knowledge 

Diagnostic 

Methods 

•  For  assessment,  measurement,  gap  identification, 
benchmarking 

Improvement 

Roadmaps 

•  To  guide  improvement  efforts  (Plan-Do-Check-Act; 
Observe-Orient-Decide-Act) 
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Types  of  Maturity  Models 


There  are  three  types  of  maturity  models 


•  Progression  Maturity  Models 

•  Capability  Maturity  Models  (CMM) 

•  Hybrid  Maturity  Models 


One  or  more  may  be  appropriate 
for  your  particular  needs 


Not  all  maturity  models  are  CMMs 
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Progression  Model  Defined 


Simple  progression  or  scaling  of  an  attribute, 
characteristic,  pattern,  or  practice 


Levels  describe  higher  states 
of  achievement,  advancement, 
completeness,  or  evolution 


Levels  can  be  agreed 
upon  by  users, 
industry,  etc. 


A  Maturity 
Progression  for  Toy 
Building  Bricks 
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Progression  Model  Example 


A  Maturity  Progression 
for  Toy  Building  Bricks 

Lego  Mindstorms 
Lego  Architecture 
Lego  Technic 
Lego  City 
Lego  Duplo 
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Progression  Model  Example  (cont.) 


A  Maturity 
Progression  for 
Counting 

Computer 

Calculator 

Adding  machine 

Slide  rule 

Abacus 

Pencil  and  paper 

Sticks/Stones 

Fingers 


A  Maturity  Progression  for 
Authentication 

Three-factor  authentication 

Two-factor  authentication 

Addition  of  changing  every  60  days 

Use  of  strong  passwords 

Use  of  simple  passwords 


Progress  does  not  necessarily  equal  process  maturity 
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Progression  Model  Example:  SGMM 


5 

4 

3 

2 

1 

0 


175  Characteristics:  Features  you 
would  expect  to  see  at  each  stage 
of  the  smart  grid  journey 


ItipttminHa  m  twi  v  « spsnsin#  w 

.g _ 


1  AiaotHRiofw 
pwrt*  E  Mttfe  i  l*  ayiiar. 

2  Ah?  ti*j  jr>  t®«:  j  nj  jr 


2  irt  a*|i  ii  mru 

Iff!  rt  K3C9  bn)  Aon 


n  swj  sot  rticsje 


1  Raws  w  jnj  nifrfttit  ®  (ift 

ngjan  rrVn  rar,  fate  jffkrtaya  d  mrirfnffira 

ilk  I 


SMR 

OS 

GO 

WAM 

TECH 

OUST 

VCI 

SE 

Strategy, 

Management, 

&  Regulatory 

Organization 
&  Structure 

Grid 

Operations 

Work  &  Asset 
Management 

Technology 

Customer 

Value  Chain 
Integration 

Societal  & 
Environmental 
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Benefits  &  Limitations  of  Progression  Models 


Benefits 

♦  Provides  a 
transformative  roadmap 

♦  Simple  to  understand 
and  us 

♦  Low  adoption  cost 

♦  Easy  to  recalibrate  as 
technologies  and 
practices  advance 


Limitations 

♦  Levels  could  be  arbitrarily 
defined 

-  Okay,  as  long  as  applied 
consistently. 

♦  Achieving  higher  levels  of 
“practice  maturity”  does  not 
necessarily  translate  into 
“process  maturity” 

♦  Often  confused  with  CMMs  - 
thus  users  inaccurately 
project  traits  of  CMMs  on 
progression  models 
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Capability  Maturity  Models  (CMM) 


A  more  complex  instrument  I 

Characterizes 

—  the  maturity  of  processes 

—  the  maturity  of  the  culture  of  the  organization  L 

—  the  degree  to  which  processes  are  institutionalized 

—  the  extent  to  which  the  organization 
demonstrates  process  maturity 


•  Levels  reflect  the  extent  to  which  a  particular 
set  of  practices  have  been  institutionalized 

—  Institutionalized  processes  are  more  likely  to  be  retained  during 
times  of  stress. 


Progression  of  Process  Institutionalization 


Software  Engineering  Institute 


Carnegie  Mellon  University 


47 


What  Do  These  Organizations  Have  in  Common? 


Customer 

Happiness 


Chain  of  Command 
Unit  Cohesion 


Customer 

Service 


Tradition 

Protection 
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Capability  Maturity  Model  Levels 


Processes  are 
acculturated, 
defined 
measured ' 
and 
governed 


r 


Practices  are  j 
performed  j 


1 - i 

Practices  are  j 

—  Level  0  - 

j  incomplete  j 

•  Incomplete 

Higher  degrees  of 
institutionalization 
translate  to  more 
stable  processes  that 

•  are  repeatable 

•  produce  consistent 
results  over  time 

•  are  retained  during 
times  of  stress 
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Examples  of  CMM  Levels 


Example  1 

Optimized 

Quantitatively  Managed 
Defined 


Managed 
Ad  hoc 


Example  2 

Externally  integrated 
Internally  integrated 
Managed 
Performed 
Initiated 


Example  3 

Shared 

Defined 

Measured 

Managed 

Planned 

Performed  but  ad  hoc 
Incomplete 
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Capability  Maturity  Model  Example 
CERT-RMM  (1  of  6) 


CERT-  R  li*  .  Vt:**  -3  I  □  1.1 


CERT  Resilience 
Management  Model 


fM/*  A  Manurin' 

Model  for 

Opmurirwiol 

fMfc Rrflicn" 


Kk h»ird  A.  Ora  Ili 
JuILl  Hr  Allen 
Uavid  W.  While 


Framework  for  managing  and 
improving  operational  resilience 


"  ..an  extensive  super-set  of  the 
things  an  organization  could  do 
to  be  more  resilient.  ” 


http://www.cert.org/resilience/ 


-  CERT-RMM  adopter 
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CERT-RMM  (2  of  6) 


Operational  Resilience  Perspective 

•  The  emergent  property  of  an  entity  that  can  continue  to 
carry  out  its  mission  in  the  presence  of  operational  stress 
and  disruption  that  does  not  exceed  its  limit 


Disruptions  come  from  realized  risk 

•  Natural  or  manmade 

•  Accidental  or  intentional 

•  Small  or  large 

•  Information  technology  or  not 

•  Cyber  or  kinetic 
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CERT-RMM  (3  of  6) 


•  Cybersecurity,  business  continuity,  IT  disaster  recovery  are 
risk  management  processes 

•  For  operational  risk  management  to  be  effective,  these 
activities  must  work  toward  the  same  goals 

•  Operational  resilience  emerges  from  effective  operational 
risk  management 


Actions  of 
people 


ft  fatal  exception 
000059F8.  The  curi 

*  Press  any  key  ti 

*  Press  CTRL ♦ALT+1 
lose  any  unsaoei 


Systems  and 
technology 
failures 


Failed  internal 
processes 


External 

events 
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CERT-RMM  (4  of  6) 


•  Most  comprehensive  framework  for  managing  and 
improving  operational  resilience 

•  Guides  implementation  and  management  of  operational 
resilience  activities 

•  Enables  and  promotes  the  convergence  of 

—  COOP,  IT  Disaster  Recovery,  Business  Continuity 
—  Information  Security,  Cybersecurity 
—  IT  Operations 
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CERT-RMM  Process  Areas  (Domains)  (5 of 6) 


Access  Management 

Asset  Definition  and  Management 

Communications 

Compliance 

Controls  Management 

Enterprise  Focus 

Environmental  Control 

External  Dependencies  Management 

Financial  Resource  Management 

Human  Resource  Management 

Identity  Management 

Incident  Management  &  Control 

Knowledge  &  Information  Management 


Measurement  and  Analysis 
Monitoring 

Organizational  Process  Definition 
Organizational  Process  Focus 
Organizational  Training  &  Awareness 
People  Management 
Resilience  Requirements  Development 
Resilience  Requirements  Management 
Resilient  Technical  Solution  Engineering 
Risk  Management 
Service  Continuity 
Technology  Management 
Vulnerability  Analysis  &  Resolution 
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CERT-RMM  Capability  Levels  (6  of  6) 


Level  3 

•  Defined 


Level  2 

•  Managed 


Level  1 

•  Performed 


Level  0 

•  Incomplete 


i  Processes  are  \ 

| 

i  acculturated,  i 
!  defined \ 
measured 
[and  governed\ 

]  Practices  are  ] 
[_  performed  ] 
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Incident  Management  &  Control:  An  Example 


Consider  the  Incident  Management  and  Control  (IMC) 
domain  from  CERT-RMM: 


•  Goal  1 

•  Goal  2 

•  Goal  3 

•  Goal  4 

•  Goal  5 


Establish  the  IMC  process 
Detect  events 
Declare  incidents 

Respond  to  and  recover  from  incidents 
Establish  incident  learning 
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Incident  Management  by  the  CMM  Levels 


Level  0 

Level  1 

Level  2 

Level  3 

Incomplete 

Performed 

Managed 

Defined 

“We  do  some 
of  the  IMC 
practices.” 

“We  do  all  of 
the  IMC 
practices.” 

“We  do  the 

IMC  practices 
AND  we  plan 
and  govern 
the  process, 
resource  it, 
train  people  to 
do  it,  monitor 
it,  etc...” 

We  do 
everything  in 
level  2  AND 
we  have  a 
defined 
process  and 
collect 

improvement 

information.” 

Institutionalization  is  cumulative 

r*  - 
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Benefits  and  Limitations  of  CMMs 


Benefits  Limitations 


•  Provides  for 
measurement  of  core 
competencies 

•  Provides  for  rigorous 
measurement  of 
capability — the  ability  to 
retain  core  competencies 
under  times  of  stress 

•  Can  provide  a  path  to 
quantitative 
measurement 
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•  Sometimes  difficult  to  understand 
and  apply;  high  adoption  cost 

•  “Maturity”  may  not  translate  into 
actual  results 

•  Potential  false  sense  of 
achievement:  achieving  high 
maturity  in  security  practices  may 
not  mean  the  organization  is 
“secure”  enough 

•  You  can  achieve  high  maturity 
ratings  in  a  capability  model  by 
institutionalizing  ineffective,  poorly- 
designed,  or  inefficient  processes. 


Compare:  Progression  vs  CMM 
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—  Level  3 

•  Defined 

—  Level  2 

•  Managed 


—  Level  1 

•  Performed 

—  Level  0 

•  Incomplete 


Core  practices 


Progression  Model 


Capability  Model 
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Distribution  of  institutionalizing 

features 


Hybrid  Models 

Combine  best  features  of  progression  and  capability 
maturity  models 

•  Allow  for  measurement  of  evolution  or  achievement  as  in 
progression  models 

•  Add  the  ability  to  measure  capability  or  institutionalization  with 
the  rigor  of  a  CMM 

Levels  reflect  both  achievement  and  capability 
Transitions  between  levels: 

•  Similar  to  a  capability  model 
(i.e.,  describe  capability  maturity) 

•  Architecturally  use  the  characteristics, 
indicators,  attributes,  or  patterns  of  a 
progression  model 
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Capability  or  “maturity”  levels 


Hybrid  Model 


Domains:  Specific  categories  of 
attributes,  characteristics,  patterns,  or 
practices  that  form  the  content  of  the 
model 


Domain  1 

Domain  2 

Domain  3 

Domain  4 

Domain  n 

Level  3 

Measured 


Level  2 

Managed 

Level  1 

Planned 

Level  0 

Incomplete 


Model  content:  Specific  attributes, 
characteristics,  patterns,  or  practices 
that  represent  practice  progression 
and  capability 


Maturity  Levels:  Defined  sets  of 
characteristics  and  outcomes,  plus 
capability  considerations 
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Maturity  Indicator  Levels 


Hybrid  Model  Example:  ES-C2M2  (1  of  3) 


10  Model  Domains:  Logical  groupings  of  cybersecurity  practices 


Electricity  Subsector  Cybersecurity 
Capability  Maturity  Model  (ES-C2M2) 
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ELECTRICITY  SUBSECTOR 

CYBERSECURITY  CAPABILITY  MATURITY  MODEL  (ES-C2M2) 


Version  1.0 

31  May  2012 


Hybrid  Model  Example:  ES-C2M2  (2  of  3) 


Level 

Name 

Characteristics 

MILO 

Not  Performed 

•  Practices  are  not  performed 

MIL1 

Initiated 

•  Initial  practices  are  performed  but  may  be  ad  hoc 

MIL2 

Performed 

Approach  characteristic: 

•  Practices  are  more  complete  or  advanced  than  at  MIL1 

Institutionalization  characteristics: 

•  Practices  are  documented 

•  Stakeholders  are  identified  and  involved 

•  Adequate  resources  are  provided  to  support  the  process 

•  Standards  or  guidelines  are  used  to  guide  practice  implementation 

MIL3 

Managed 

Approach  characteristic: 

•  Practices  are  more  complete  or  advanced  than  at  MIL2 

Institutionalization  characteristics: 

•  Activities  are  guided  by  policy  (or  other  directives)  and  governance 

•  Policies  include  compliance  requirements  for  specified  standards  or  guidelines 

•  Activities  are  periodically  reviewed  for  conformance  to  policy 

•  Responsibility  and  authority  for  practices  are  assigned  to  personnel 

•  Personnel  performing  the  practice  have  adequate  skills  and  knowledge 

Hybrid  Model  Example:  ES-C2M2  (3  of  3) 
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Benefits  and  Limitations  of  Hybrid  Models 


Benefits 

•  Provides  for  easy 
measurement  of  core 
competencies  as  well  as 
approximation  of 
capability 

•  Can  adapt  easily  to 
evolution  of  technologies 
and  practices  without 
sacrificing  capability 
measurement 

•  Low  adoption  cost 


Limitations 

•  “Maturity”  concept  is 
approximated;  not  as 
rigorous  as  CMM 

•  Combination  of  attributes 
with  institutionalizing 
features  at  each  level  can 
be  arbitrary 

-  Okay,  as  long  as  applied 
consistently. 
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Comparison  of  Frameworks 

Progression  MM 

Capability  MM 

Hybrid  MM 

Code  of  Practice 

Other 

Smart  Grid  Maturity  Model  (SGMM) 

X 

Versions  of  COBIT  Prior  to  Version  5 

X 

Building  Security  In  Maturity  Model  (BSIMM) 

X 

Gartner  ITScore  for  Infrastructure  and  Operations 

X 

Forrester  Information  Security  Maturity  Model 

X 

CMMI  Resilience 

X 

CERT®  Resilience  Management  Model  (CERT-RMM) 

X 

COBIT  Version  5 

X 

Software  Assurance  Maturity  Model  (SAMM) 

X 

The  Open  Group  Info.  Security  Management  Maturity  Model  (0-ISM3) 

X 

Electricity  Subsector  Cybersecurity  Maturity  Model  (ES-C2M2) 

X 

Oil  &  Natural  Gas  Cybersecurity  Maturity  Model  (ONG-C2M2) 

X 

Some  framework  based  on  ISO  27000  family  of  standards 

X 

Information  Security  Forum  Standard  of  Good  Practice  for  Info.  Security 

X 

NIST  Framework  for  Improving  Critical  Infrastructure  Cybersecurity 

X 
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Maturity  Models  Member  Query  -  Q5 
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Which  maturity  models? 
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i . 


Some  framework  based  on  ISO  27000  family  of  standards 

CMMI 

NIST  Framework  for  Improving  Critical  Infrastructure  Cybersecurity 

Other  (Please  Specify): 
Maturity  models  from  Gartner  and/or  Forrester  Research 
Information  Security  Forum  Security  Model 
An  internally  developed  maturity  model 
Electricity  Subsector  Cybersecurity  Maturity  Model  (ES-C2M2) 
Oil  &  Natural  Gas  Cybersecurity  Maturity  Model  (ONG-C2M2) 
Building  Security  In  Maturity  Model  (BSIMM) 
CERT  Resilience  Management  Model  (RMM) 
Smart  Grid  Maturity  Model  (SGMM) 
Software  Assurance  Maturity  Model  (SAMM) 
The  Open  Group  Inf.  Security  Mmgt.  Maturity  Model  (0-ISM3) 
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Maturity  Models  Member  Query  -  Q5 

Which  maturity  models? 
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OTHER: 

•  WEF 

•  COBIT 

•  COBIT 

•  COBIT 

•  Proprietary 

•  A  blend  of  several 

•  SANS  top  20  critical  controls 

•  HMG  Information  Assurance  Maturity  Model 

•  Internally  developed  model  based  on  COBIT 
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Maturity  Models  Member  Query  -  Q6 


If  no,  how  do  you  assess  the  maturity  of  your 
cybersecurity  program? 


•  In  an  ad  hoc  manner 

•  Best  of  breed  analytics 

•  We  are  intending  to  use  an  external  consultancy  that 
benchmarks  to  the  NIST  Cybersecurity  framework. 
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Panel  Discussion 


•  Real-life  Examples 

•  Success  Stories 

•  Lessons  Learned 

•  Recommendations 


Planned  Members'  Opening  Remarks 


Ben 

Krutzen 

Jason 

Christopher 

David 

White 

Shell 


U.S.  Department 
of  Energy 


Axio  Global 


Question/Answer  Session  with  the  Panel 


Ben 

Krutzen 

Jason 

Christopher 

David 

White 

Shell 


U.S.  Department 
of  Energy 


Axio  Global 


Closing  Thoughts 

•  Summary 

•  A  few  cautions 

•  Determining  when  and  which  type  to  use 


First  and  Foremost 


•  Have  a  clear  understanding  of  your  business  objectives  for 
using  any  type  of  improvement  model 

—  How  the  model  will  meet  these  objectives 

•  Understand  how  this  initiative  fits  with  others  that  are 
mainstream  for  the  organization  (not  a  new  add-on) 

•  Have  visible  sponsorship  of  executives  and  senior  leaders 
who  are  essential  for  success 

•  Have  well-defined  outcome  measures  that  are  regularly 
reported  and  reviewed 

•  Have  a  plan  and  committed  resources 
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A  Few  Cautions 


Progression  models  may  be  easier  to  adopt  but 
may  not  be  sustainable  (aka  sticky) 

Definitions  of  levels  can  be  arbitrary 

•  and,  therefore,  important  to  ensure  consistency  over 
time  and/or  over  instances  of  being  applied 

Measuring  process  performance  and  maturity  is 
useful  but  may  not  be  sufficient 

Exercise  care  when  using  maturity 
models  for  specific  purposes 
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Progression  Models  May  Not  Be  Sustainable 


A  progression  model  provides  a  roadmap 
or  scale  of  a  particular  characteristic, 
indicator,  attribute,  pattern,  or  practice 


•  Focuses  on  practices  or  controls  and  their  progression  from  least 
mature  to  most  mature 


•  Cannot  be  used  to  measure  the  extent  to  which  an  organization  is 
capable  of  sustaining  the  practice  in  times  of  disruption  and  stress 
(the  practice  has  not  become  part  of  the  DNA) 


A  hybrid  or  capability  maturity  model  adds  the  dimension 
of  organizational  capability  to  practice  progression 

•  Thus  able  to  measure  an  organization’s 
“resilience”  in  the  presence  of  disruption 
and  stress 
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Definitions  of  Levels  and  the  Scale 


Often  defined  by  consensus  of  subject  matter  experts 

Can  simply  reflect  a  plateau  or  a  place  in  a  progression  or  scale 

Often  have  not  been  validated  or  are  difficult  to  validate  based 
on  experience  and  measurement 

May  neglect  to  represent  the  capability  and  capacity  of  an 
organization  to  sustain  operations  in  the  presence  of  disruption 
and  stress 

Arbitrarily  defined  levels  are  fine  so  long  as  the 
scale  is  applied  consistently: 

•  over  time  (e.g.,  to  measure  improvement) 

•  over  instances  (e.g.,  for  benchmarking) 
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Measuring  Process  Performance  May  Not 
Be  Sufficient 

Experience  demonstrates  that  the  quality  of  the  process 
directly  affects  the  quality  of  the  product 

•  However,  process  performance  and  maturity  are  only  one  aspect 

Also  need  to  consider  the  performance  and  maturity  of 

•  The  product  and  its  outcomes 

•  The  supporting  technologies 

•  The  environment  within  which  the  product  operates 

•  Knowledge,  skills,  and  abilities  of  people  with 
respect  to  all  of  these 

•  Which  of  these  dimensions  to  emphasize 
given  product  objectives 

You  can  achieve  high  maturity  ratings  in  a 
capability  model  by  institutionalizing  ineffective, 
poorly-designed,  or  inefficient  processes. 
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When  Does  It  Make  Sense  to  Use  Maturity 
Models? 

Requirement  for  a  structured  approach 

Demonstrated,  measurable  results  based  on  an 
established  body  of  knowledge 

A  defined  roadmap  from  a  current  state  to  a  desired 
state 

An  ability  to  monitor  and  measure  progress, 
particularly  in  the  presence  of  change 

•  Response  to  a  strategic  improvement  or  new 
product/new  market  objective 
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When  Does  It  Make  Sense  to  Use  Maturity 
Models?  (cont.) 

Desire  to  answer  these  questions  in  a  repeatable, 
predictable  manner: 

•  How  do  I  compare  with  my  peers?  (ability  to  benchmark) 

•  How  can  I  determine  how  secure  I  am  and  if  I  am  secure  enough? 

•  How  do  I  measure  my  current  state?  Characterize  my  desired  state? 

•  What  concrete  actions  do  I  need  to  take  to  improve?  And  in  what 
order? 

•  How  do  I  measure  progress  toward  my  desired  state? 

•  How  do  I  adapt  to  change? 
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Exercise  Care  When  Using  Maturity  Models 


If  the  immediate  need  is  to  respond  to  an  in-progress 
disruptive  event 

•  Robust  processes  are  not  yet  in  place 

•  Current  protection  and  defensive  mechanisms  are  failing 

•  Need  to  stop  the  bleeding,  stabilize  operations,  rely  on  experts 


In  response  to  current  and  new  compliance  requirements 


•  In  a  highly  regulated  industry 

•  Must  demonstrate  compliance  with  specific 
laws,  regulations  and  standard(s) 

•  Standard,  defined  processes  and  mapping 
new  compliance  requirements  to  these  can 
be  quite  effective 
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Thank  you  for  your  attention... 
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